Fully automated SEM sampling system for e-beam image enhancement

ABSTRACT

Disclosed herein is a method of automatically obtaining training images to train a machine learning model that improves image quality. The method may comprise analyzing a plurality of patterns of data relating to a layout of a product to identify a plurality of training locations on a sample of the product to use in relation to training the machine learning model. The method may comprise obtaining a first image having a first quality for each of the plurality of training locations, and obtaining a second image having a second quality for each of the plurality of training locations, the second quality being higher than the first quality. The method may comprise using the first image and the second image to train the machine learning model.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority of U.S. application 62/787,031 whichwas filed on Dec. 31, 2018, and which is incorporated herein in itsentirety by reference.

FIELD

The present disclosure relates generally to systems for imageacquisition and image enhancement methods, and more particularly, tosystems for and methods of improving metrology by automaticallyobtaining training images to train a machine learning model thatimproves image quality.

BACKGROUND

In manufacturing processes used to make integrated circuits (ICs),unfinished or finished circuit components are inspected to ensure thatthey are manufactured according to design and are free of defects.Inspection systems utilizing optical microscopes or charged particle(e.g., electron) beam microscopes, such as a scanning electronmicroscope (SEM), can be employed. As the physical sizes of ICcomponents continue to shrink, accuracy and yield in defect detectionbecome more and more important. However, imaging resolution andthroughput of inspection tools struggle to keep pace with theever-decreasing feature size of IC components. Further improvements inthe art are desired.

SUMMARY

The following presents a simplified summary of one or more aspects inorder to provide a basic understanding of such aspects. This summary isnot an extensive overview of all contemplated aspects, and is intendedto neither identify key or critical elements of all aspects nordelineate the scope of any or all aspects. Its sole purpose is topresent some concepts of one or more aspects in a simplified form as aprelude to the more detailed description that is presented later.

In an aspect of the disclosure, there is provided a method ofautomatically obtaining training images to train a machine learningmodel. The method may comprise analyzing a plurality of patterns of datarelating to a layout of a product to identify a plurality of traininglocations on a sample of the product to use in relation to training themachine learning model. The method may comprise obtaining a first imagehaving a first quality for each of the plurality of training locations,and obtaining a second image having a second quality for each of theplurality of training locations, the second quality being higher thanthe first quality. The method may comprise using the first image and thesecond image to train the machine learning model.

In another aspect of the disclosure, there is provided an apparatus forautomatically obtaining training images to train a machine learningmodel. The apparatus may comprise a memory, and one or more processorscoupled to the memory. The processor(s) may be configured to analyze aplurality of patterns of data relating to a layout of a product toidentify a plurality of training locations on a sample of the product touse in relation to training the machine learning model. The processor(s)may be further configured to obtain a first image having a first qualityfor each of the plurality of training locations, and obtain a secondimage having a second quality for each of the plurality of traininglocations, the second quality higher than the first quality. Theprocessor(s) may be further configured to use the first image and thesecond image to train the machine learning model.

In another aspect of the disclosure, there is provided a non-transitorycomputer readable medium storing a set of instructions that isexecutable by a controller of a device to cause the device to perform amethod comprising: analyzing a plurality of patterns of data relating toa layout of a product to identify a plurality of training locations on asample of the product to use in relation to training the machinelearning model; obtaining a first image having a first quality for eachof the plurality of training locations; obtaining a second image havinga second quality for each of the plurality of training locations, thesecond quality higher than the first quality; and using the first imageand the second image to train the machine learning model.

In another aspect of the disclosure, there is provided an electron beaminspection apparatus comprising a controller having circuitry to causethe electron beam inspection apparatus to perform: analyzing a pluralityof patterns of data relating to a layout of a product to identify aplurality of training locations on a sample of the product to use inrelation to training the machine learning model; obtaining a first imagehaving a first quality for each of the plurality of training locations;obtaining a second image having a second quality for each of theplurality of training locations, the second quality being higher thanthe first quality; and using the first image and the second image totrain the machine learning model.

To accomplish the foregoing and related ends, aspects of embodimentscomprise the features hereinafter described and particularly pointed outin the claims. The following description and the annexed drawings setforth in detail certain illustrative features of the one or moreaspects. These features are indicative, however, of but a few of thevarious ways in which the principles of various aspects may be employed,and this description is intended to include all such aspects and theirequivalents.

BRIEF DESCRIPTION OF FIGURES

FIG. 1 is a flow diagram of a process for improving images obtained by aSEM sampling system.

FIG. 2 is a block diagram illustrating an example of an automatic SEMsampling system, according to some aspects of the present disclosure.

FIG. 3 is a schematic diagram illustrating an example of an electronbeam inspection (EBI) system, according to some aspects of the presentdisclosure.

FIG. 4 is a schematic diagram illustrating an example of an electronbeam tool that can be a part of the example electron beam inspection(EBI) system of FIG. 3 , according to some aspects of the presentdisclosure.

FIGS. 5A-5C illustrate a plurality of design patterns of a graphicdatabase system (GDS) of a product, according to some aspects of thepresent disclosure.

FIG. 5D is a diagram illustrating a plurality of training locations,according to some aspects of the present disclosure.

FIG. 6A is a flow diagram illustrating an example of a method ofautomatically obtaining training images to train a machine learningmodel, according to some aspects of the present disclosure.

FIG. 6B is a block diagram illustrating details of an automatic SEMsampling system, according to some aspects of the present disclosure.

FIG. 7 is a block diagram illustrating an example of a method ofautomatically obtaining training images to train a machine learningmodel that improves image quality, according to some aspects of thepresent disclosure.

DETAILED DESCRIPTION

Reference will now be made in detail to example aspects of embodiments,examples of which are illustrated in the accompanying drawings. Thefollowing description refers to the accompanying drawings in which thesame numbers in different drawings represent the same or similarelements unless otherwise represented. The implementations set forth inthe following description of example aspects of embodiments do notrepresent all implementations consistent with the invention. Instead,they are merely examples of apparatuses and methods consistent withaspects of embodiments related to the invention as recited in theclaims. For example, although some aspects of embodiments are describedin the context of utilizing electron beam inspection (EBI) system suchas scanning electron microscope (SEM) for generation of a wafer image,the disclosure is not so limited. Other types of inspection system andimage generation system be similarly applied.

The enhanced computing power of electronic devices, while reducing thephysical size of the devices, can be accomplished by significantlyincreasing the packing density of circuit components such as,transistors, capacitors, diodes, etc. on an IC chip. For example, in asmart phone, an IC chip (which is the size of a thumbnail) may includeover 2 billion transistors, the size of each transistor being less than1/1000^(th) of a human hair. Not surprisingly, semiconductor ICmanufacturing is a complex process, with hundreds of individual steps.Errors in even one step have the potential to dramatically affect thefunctioning of the final product. Even one “killer defect” can causedevice failure. The goal of the manufacturing process is to improve theoverall yield of the process. For example, for a 50-step process to get75% yield, each individual step must have a yield greater than 99.4%,and if the individual step yield is 95%, the overall process yield dropsto 7%.

In various steps of the semiconductor manufacturing process, patterndefects can appear on at least one of a wafer, a chip, or a mask, whichcan cause a manufactured semiconductor device to fail, thereby reducingthe yield to a great degree. As semiconductor device sizes continuallybecome smaller and smaller (along with any defects), identifying defectsbecomes more challenging and costly. Currently, engineers insemiconductor manufacturing lines spend usually hours (and evensometimes days) to identify locations of small detects to minimize theirimpact on the final product.

Conventional optical inspection techniques are ineffective in inspectingfor small defects (e.g., nanometer scale defects). Advancedelectron-beam inspection (EBI) tools, such as an SEM with highresolution and large depth-of-focus, have been developed to meet theneed in the semiconductor industry. E-beam images may be used inmonitoring semiconductor manufacturing processes, especially for moreadvanced nodes where optical inspection falls short of providing enoughinformation.

An E-beam image may be characterized according to one or more qualities,such as contrast, brightness, noise level, etc. In general, a lowquality images often requires less parameter tuning and fewer scans, butthe information embedded in the low quality images (such as defect typesand locations) is hard to extract, which may have a negative impact onthe analyses. High quality images which do not suffer from this problemmay be obtained by an increased number of scans. However, high qualityimages may have a low throughput.

Further, an E-Beam image acquisition procedure may go through manysteps, such as identifying the pattern of interests, setting up scanningareas for inspection, tuning SEM conditions, determining qualityenhancement methods, etc. Many of these settings and parameters arecontributing factors to both the system throughput and the E-beam imagequality. There may be a trade-off between the throughput and imagequality.

In order to obtain high quality images and at the same time achieve ahigh throughput, an operator generally needs to set many parameters andmake decisions as to how the images should be obtained. However,determining these parameters is often not straightforward. To minimizepossible operator-to-operator variation, machine learning basedenhancement methods can be trained to learn the enhancementframework/network. In such cases, the acquisition of sufficient andrepresentative training samples is advantageous to increase the finalperformance of the trained system. However, common procedures to obtainSEM image samples require significant human intervention, includingsearching the proper design patterns for scanning, determining a numberof images to collect and various imaging conditions, etc. Such intensivehuman involvement impedes full utilization of the advanced machinelearning based enhancement methods. Therefore, there is a need todevelop a fully automated smart sampling system for E-beam imagesquality enhancement.

Disclosed herein, among other things, is equipment that automaticallyobtains training images to train a machine learning model that improvesimage quality, and methods used by the equipment. Electron-beam (E-beam)imaging plays an important role in inspecting very small defects (e.g.,nanometer scale defects with a nanometer bring 0.000000001 meters) insemiconductor manufacturing processes. In general, it is possible toobtain many relatively low quality E-beam images very quickly, but theimages may not provide enough useful information about issues such asdefect types and locations. On the other hand, it is possible to obtainhigh quality images but doing so takes more time and so decreases thespeed at which devices can be analyzed. This increases manufacturingcosts. Samples may be scanned multiple times to improve image quality,such as by reducing noise by averaging over multiple images. However,scanning each sample multiple times reduces the system throughput andmay cause charge build up or may damage the sample.

Some of the systems and methods disclosed herein embody ways to achievehigh quality images with a reduced number of scans of a sample, in someembodiments by automatically selecting sample locations to use togenerate training images for a machine learning (ML) algorithm. The term“quality” refers to a resolution, a contrast, a sensitivity, abrightness, or a noise level, etc. Some of the systems and methodsdisclosed herein may obtain the benefit of higher quality images withoutexcessively slowing down production. Some embodiments of the systems mayautomatically analyze a plurality of patterns of data relating to alayout of a product, in order to identify multiple training locations ona sample of the product to use in relation to training a machinelearning model. The patterns of data may be SEM images of the product,or a layout design (e.g., graphics database system (GDS), Open ArtworkSystem Interchange Standard (OASIS), Caltech Intermediate Format (CIF),etc.) representation of a product). As an example, a GDS analyzer may beused to determine good locations on a sample to use for training the MLalgorithm. These locations are scanned multiple times, resulting inasset of images, some of which incrementally improve with each scan,such as due to noise averaging or using a higher resolution setting onthe SEM. These images (e.g., lower quality images and associated higherquality images) are used as training samples to train the ML algorithm.Other locations on the sample are scanned a reduced number of times, andthe ML algorithm modifies the image to approximate how it wouldincrementally improve with additional scans or with a scan using ahigher resolution setting.

The disclosure provides, among others, a method of automaticallyobtaining training images to train a machine learning model thatimproves image quality. The method may comprise analyzing a plurality ofpatterns of data relating to a layout of a product to identify aplurality of training locations on a sample of the product to use inrelation to training the machine learning model. For example, the methodmay comprise analyzing a plurality of patterns from a graphic databasesystem (GDS) of a product, and identifying a plurality of traininglocations on a sample of the product to use in relation to training themachine learning model based on the analyzing.

As an example, the method may obtain one or more low quality images andone or more high quality images for each of the multiple traininglocations. The method may use the low quality image(s) and the highquality image(s) to train the machine learning model. For example, themachine learning model can learn how an image changes between a lowquality image and a high quality image, and can be trained to generatean image that approximates a high quality image from a low qualityimage. After the training, the machine learning model may be used toautomatically generate high quality images from low quality images forthe product. In this way, the high quality images may be obtainedquickly. Further, the method may minimize the amount of humansupervision required, prevent inconsistency that would otherwise resultfrom used by different operators, and avoid various human errors.Therefore, the method may increase inspection accuracy. Accordingly, themethod may increase manufacturing efficiency and reduce themanufacturing cost.

As an example, the method may further comprise using the machinelearning model to modify an image to approximate a result obtained withan increased number of scans. The term “approximate” refers to comeclose or be similar to in quality. For example, a quality of an imageusing the machine learning model may be within 5%, 10%, 15%, or 20% of aquality from an image obtained with an increased number of scans.

Some of the methods disclosed herein are advantageous for generatinghigh quality images with high throughput. Further, some of the methodsmay require minimal human intervention, reducing or eliminatinginconsistency from different operators and various human errors, andthereby increasing the accuracy of the inspection. In this way,manufacturing efficiency may be increased and the manufacturing cost maybe reduced.

Some disclosed embodiments provide a fully automated smart E-beam imagesampling system for E-beam image enhancement, which comprises a GDSpattern analyzer and a smart inspection sampling planner for collectingsample images to feed into a machine learning quality enhancement systemto generate non-parameterized quality enhancement modules. The automatedsmart E-beam image sampling system can be used to enhance lower qualityimages collected from higher throughput mode. The automated smart E-beamimage sampling system is advantageous to require minimum humanintervention and generate high quality images with high throughput forinspection and metrology analysis. Advantageously, the automated smartE-beam image sampling system can increase manufacturing efficiency andreduce manufacturing cost.

FIG. 1 is a flow diagram 100 of a process for improving images obtainedby SEM sampling system 102, according to some aspects of the disclosure.The automated smart SEM sampling system 102 may, for example, be EBIsystem 300 of FIG. 3 . The SEM sampling system 102 may be configured toobtain training images to train a machine learning model that improvesimage quality. The automated smart SEM sampling system 102 may beconfigured to analyze a plurality of patterns of data relating to alayout of a product to identify a plurality of training locations on asample of the product to use in relation to training the machinelearning model. For example, the data may be in a database. For example,the database may be any one of a GDS, an Open Artwork System InterchangeStandard, or a Caltech Intermediate Form. For example, the GDS mayinclude both GDS and GDSII. Further, the automated smart SEM samplingsystem 102 may comprise a GDS pattern analyzer that is configured toanalyze a plurality of patterns from a graphic database system (GDS) 101of a product. The automated smart SEM sampling system 102 mayadditionally comprise a smart inspection sampling planner that isconfigured to identify a plurality of training locations on a sample ofthe product to use in relation to training the machine learning modelbased on the analyzing.

The automated smart SEM sampling system 102 may be configured to obtaina first image having a first quality for each of the plurality oftraining locations. For example, the system 102 may be configured toenable a first scan for each of the plurality of training locations toobtain a first image for each of the plurality of training locations.The automated smart SEM sampling system 102 may be configured to obtainthe first image for each of the plurality of training locations based onthe first scan. Further, the system 102 may be configured to obtain morethan one first image having a first quality for each of the plurality oftraining locations. For example, the first scan may include a low numberof scans and the first image may be a low quality image. The low numberof scans may be a number of scans in a range of about, e.g., 1 to about10. The automated smart SEM sampling system 102 may be configured toobtain a second image for each of the plurality of training locations,wherein the second image has a second quality higher than the firstquality. For example, the second image may be a high quality image. Thesecond image may be a higher quality image due to having a higherresolution, a higher contrast, a higher sensitivity, a higherbrightness, or a lower noise level, etc., or some combination of these.

For example, the system 102 may be configured to obtain more than onesecond image having a second quality for each of the plurality oftraining locations. As an example, the second image may be obtained byenabling a second set (or series) of scans, where the set (or series) ofscans can include an increased number of scans, thereby resulting in ahigher quality image. For example, the increased number of scans may bea number of scans in the range of about 32 to about 256. As anotherexample, a higher quality image may be obtained by taking a number oflow quality images, and averaging the images, to result in a higherquality image (due to, e.g., less noise as a result of the averaging).As still another example, a higher quality image may be obtained bycombining a number of low quality images to result in a higher qualityimage. As yet another example, the second image may be received as areference image by an optional user input, as illustrated at 105. As onemore example, the second image may be obtained based on an improvedquality scan, such as a scan with a higher resolution or other settingchange(s) that result in an improved quality scan.

The automated smart SEM sampling system 102 may use the first image andthe second image for each of the plurality of training locations astraining images to train the machine learning model. In some aspects,the automated smart SEM sampling system 102 may be configured to enablescanning each of the training locations a plurality of times to create aplurality of training images for each location, where some of thetraining images reflect an improvement in image quality that resultsfrom an additional number of scans of a training location. For example,a plurality of low quality image and high quality image pairs may beobtained and used as training images to train the machine learningmodel. For example, the automated smart SEM sampling system 102 maycollect sample images (e.g., the training image pairs) to feed into amachine learning based quality enhancement system 107 to generatenon-parameterized quality enhancement modules. The automated smart SEMsampling system can be used to enhance low quality images 108 collectedduring operation in a high throughput mode to generate enhanced highquality images 109. For example, the automated smart SEM sampling system102 may be configured to use the machine learning model to modify animage to approximate a result obtained with an increased number ofscans. The fully automated smart SEM sampling system 102 has theadvantage of requiring a minimal amount of human intervention whilebeing able to generate high quality images with high throughput forinspection and metrology analysis.

FIG. 2 is a block diagram 200 illustrating an example of an automatedSEM sampling system 202, according to some aspects of the presentdisclosure. As shown in FIG. 2 , the automated SEM sampling system 200may comprise a computer system 202 (e.g., computer system 309 in FIG. 3), which is in communication with an inspection system 212 and areference storage device 210. For example, the inspection system 212 maybe an EBI tool (e.g., EBI system 300 of FIG. 3 ). The computer system202 may comprise a processor 204, a storage medium 206 and a userinterface 208. The processor 204 can comprise multiple processors, andthe storage medium 206 and the reference storage device 210 can be asame single storage medium. The computer system 202 may be incommunication with the inspection system 212 and the reference storagedevice 210 via wired or wireless communications. For example, thecomputer system may be a controller of the EBI tool, and the controllermay have circuitry to cause the EBI tool to perform automated SEMsampling.

The computer system 202 may include, but is not limited to, a personalcomputer, a workstation, a network computer or any device having one ormore processors. The storage medium 206 stores SEM sampling instructionsand the processor 204 is configured (via its circuitry) to execute theSEM sampling instructions to control the automated SEM sampling process.The processor 204 may be configured to obtain training images to train amachine learning model that improves image quality, as described inconnection with FIG. 1 . For example, the processor 204 may beconfigured to analyze a plurality of GDS patterns of a product andidentify a plurality of training locations on a sample of the product.The processor 204 may communicate with the inspection system 212 toenable a first scan for each of the plurality of training locations toobtain a first image for each of the plurality of training locations.For example, the processor 204 may instruct the inspection system 212 toperform the first scan to obtain the at least one image, which may be alow quality image with a lower number of scans. The processor 204 mayobtain the first image for each of the plurality of training locationsbased on the first scan from the inspection system 212. The processor204 may further obtain a second image, which may be a high qualityimage, for each of the plurality of training locations. For example, theprocessor 204 may instruct the inspection system 212 to perform a secondscan with an increased number of scans to obtain the high quality image.For another example, the processor 204 may obtain the high quality imageas a reference image from reference storage device 210, by an optionaluser input. The processor 202 may be configured to use the first image(e.g., low quality image) and the second image (e.g., high qualityimage) for each of the plurality of training locations as trainingimages to train the machine learning model. In some aspects, a pluralityof low quality image and high quality image pairs may be obtained andused as training images to train the machine learning model. Theprocessor 204 may be further configured to use the machine learningmodel to modify a first image of a new location with a low quality andgenerate a high quality image of the new location.

The user interface 208 may include a display configured to display animage of a wafer, an input device configured to transmit user command tocomputer system 202, etc. The display may be any type of a computeroutput surface and projecting mechanism that shows text and graphicimages, including but not limited to, cathode ray tube (CRT), liquidcrystal display (LCD), light-emitting diode (LED), gas plasma, a touchscreen, or other image projection technologies, for displayinginformation to a computer user. The input device may be any type of acomputer hardware equipment used to provide data and control signalsfrom an operator to computer system 202. The input device may include,but is not limited to, a keyboard, a mouse, a scanner, a digital camera,a joystick, a trackball, cursor direction keys, a touchscreen monitor,or audio/video commanders, etc., for communicating direction informationand command selections to processor or for controlling cursor movementon display.

The reference storage device 210 may store a reference file databasethat is accessed by computer system 202 during the automated SEMsampling process. In some embodiments, reference storage device 210 maybe a part of computer system 202. The reference image file forinspection of the wafer can be manually provided to computer system 202by a human operator. Alternatively, reference storage device 210 may beimplemented with a processor and the reference image file can beautomatically provided to computer system 202 by reference storagedevice 210. Reference storage device 210 may be a remote server computerconfigured to store and provide any reference images, may be cloudstorage, etc.

Inspection system 212 can be any inspection system that can generate animage of a wafer. For example, the wafer can be a sample of the product,of which the plurality of design patterns of the GDS is analyzed by theprocessor 204. The wafer can be a semiconductor wafer substrate, asemiconductor wafer substrate having one or more epitaxial layers orprocess films, etc. The embodiments of the present disclosure are notlimited to use in a specific type for wafer inspection system 212 aslong as the wafer inspection system can generate a wafer image having aresolution high enough to observe key features on the wafer (e.g., lessthan 20 nm), consistent with contemporary semiconductor foundrytechnologies. In some aspects of the present disclosure, inspectionsystem 212 is an electron beam inspection (EBI) system 304 describedwith respect to FIG. 3 .

Once a wafer image is acquired by inspection system 212, the wafer imagemay be transmitted to computer system 202. Computer system 202 andreference storage device 210 may be part of or remote from inspectionsystem 212.

In some aspects of embodiments, the automated SEM sampling system 202may further comprise the inspection system 212 and the reference storagedevice 210. For example, the automated SEM sampling system 202 may befurther configured to perform a first scan for each of the plurality oftraining locations to obtain at least one first image for each of theplurality of training locations. For another example, the automated SEMsampling system 202 may be further configured to perform a second scanfor each of the plurality of training locations to obtain the at leastone second image for each of the plurality of training locations. Forexample, the at least one second image may have an enhanced qualityresulted from an increased number of scans.

FIG. 3 is a schematic diagram illustrating an example electron beaminspection system, according to some aspects of the present disclosure.As shown in FIG. 3 , electron beam inspection system 300 includes a mainchamber 302, a load/lock chamber 304, an electron beam tool 306, acomputer system 309, and an equipment front end module 308. The computersystem 309 may be a controller of the electron beam inspection system300. Electron beam tool 306 is located within main chamber 302.Equipment front end module 308 includes a first loading port 308 a and asecond loading port 308 b. Equipment front end module 308 may includeadditional loading port(s). First loading port 308 a and second loadingport 308 b receive wafer cassettes that contain wafers (e.g.,semiconductor wafers or wafers made of other material(s)) or samples tobe inspected (wafers and samples are collectively referred to as“wafers” hereafter). One or more robot arms (not shown) in equipmentfront end module 308 transport the wafers to load/lock chamber 304.Load/lock chamber 304 is connected to a load/lock vacuum pump system(not shown) which removes gas molecules in load/lock chamber 304 toreach a first pressure below the atmospheric pressure. After reachingthe first pressure, one or more robot arms (not shown) transport thewafer from load/lock chamber 304 to main chamber 302. Main chamber 302is connected to a main chamber vacuum pump system (not shown) whichremoves gas molecules in main chamber 302 to reach a second pressurebelow the first pressure. After reaching the second pressure, the waferis subject to inspection by electron beam tool 306. The electron beamtool 306 may scan a location a plurality of times to obtain an image. Ingeneral, a low quality image may be obtained by a low number of scanswith a high throughput, and a high quality may be obtained by a highnumber of scans with a low throughput.

FIG. 4 is a schematic diagram illustrating an example of an electronbeam tool 400 (e.g., 306) that can be a part of the example electronbeam inspection system of FIG. 3 , according to some aspects of thepresent disclosure. FIG. 4 illustrates examples of components ofelectron beam tool 306, according to some aspects of the presentdisclosure. As shown in FIG. 4 , the electron beam tool 400 may includea motorized stage 400, and a wafer holder 402 supported by motorizedstage 400 to hold a wafer 403 to be inspected. Electron beam tool 400further includes an objective lens assembly 404, electron detector 406(which includes electron sensor surfaces), an objective aperture 408, acondenser lens 410, a beam limit aperture 412, a gun aperture 414, ananode 416, and a cathode 418. Objective lens assembly 404, in someaspects, can include a modified swing objective retarding immersion lens(SORIL), which includes a pole piece 404 a, a control electrode 404 b, adeflector 404 c, and an exciting coil 404 d. The electron beam tool 400may additionally include an energy dispersive X-ray spectrometer (EDS)detector (not shown) to characterize the materials on the wafer.

A primary electron beam 420 is emitted from cathode 418 by applying avoltage between anode 416 and cathode 418. Primary electron beam 420passes through gun aperture 414 and beam limit aperture 412, both ofwhich can determine the size of electron beam entering condenser lens410, which resides below beam limit aperture 412. Condenser lens 410focuses primary electron beam 420 before the beam enters objectiveaperture 408 to set the size of the electron beam before enteringobjective lens assembly 404. Deflector 404 c deflects primary electronbeam 420 to facilitate beam scanning on the wafer. For example, in ascanning process, deflector 404 c can be controlled to deflect primaryelectron beam 420 sequentially onto different locations of top surfaceof wafer 403 at different time points, to provide data for imagereconstruction for different parts of wafer 403. Moreover, deflector 404c can also be controlled to deflect primary electron beam 420 ontodifferent sides of wafer 403 at a particular location, at different timepoints, to provide data for stereo image reconstruction of the waferstructure at that location. Further, in some aspects, anode 416 andcathode 418 may be configured to generate multiple primary electronbeams 420, and electron beam tool 400 may include a plurality ofdeflectors 404 c to project the multiple primary electron beams 420 todifferent parts/sides of the wafer at the same time, to provide data forimage reconstruction for different parts of wafer 203.

Exciting coil 404 d and pole piece 404 a generate a magnetic field thatbegins at one end of pole piece 404 a and terminates at the other end ofpole piece 404 a. A part of wafer 403 being scanned by primary electronbeam 420 can be immersed in the magnetic field and can be electricallycharged, which, in turn, creates an electric field. The electric fieldreduces the energy of impinging primary electron beam 420 near thesurface of the wafer before it collides with the wafer. Controlelectrode 404 b, being electrically isolated from pole piece 404 a,controls an electric field on the wafer to prevent micro-arching of thewafer and to ensure proper beam focus.

A secondary electron beam 422 can be emitted from the part of wafer 403upon receiving primary electron beam 420. Secondary electron beam 422can form a beam spot on a surface of a sensor of electron detector 406.Electron detector 406 can generate a signal (e.g., a voltage, a current,etc.) that represents an intensity of the beam spot and provide thesignal to a processing system (not shown). The intensity of secondaryelectron beam 422, and the resultant beam spot, can vary according tothe external or internal structure of wafer 403. Moreover, as discussedabove, primary electron beam 420 can be projected onto differentlocations of the top surface of the wafer to generate secondary electronbeams 422 (and the resultant beam spot) of different intensities.Therefore, by mapping the intensities of the beam spots with thelocations of wafer 403, the processing system can reconstruct an imagethat reflects the internal or external structures of wafer 403. Once awafer image is acquired by electron beam tool 400, the wafer image maybe transmitted to computer system 402 (e.g., 202, as shown in FIG. 2 ).

FIGS. 5A-5C illustrate a plurality of design patterns of a database suchas a GDS database of a product, according to some aspects of the presentdisclosure. The automated SEM sampling system disclosed herein may beconfigured to perform a method of automatically obtaining trainingimages to train a machine learning model that improves image quality.For example, the automated SEM sampling system may be a controller ofthe EBI tool, and the controller may have circuitry to cause the EBItool to perform automated SEM sampling. For example, the automated SEMsampling system may comprise a GDS analyzer (e.g., a GDS analyzercomponent). The GDS analyzer may be configured to perform patternanalysis and classification based on various features, i.e. linepattern, logic pattern, 1D/2D pattern, dense/isolated pattern, etc. Samepatterns may be grouped together via pattern grouping.

For example, a plurality of manufacture design patterns may be renderedfrom a GDS input. At this stage, the plurality of patterns are scatteredpatterns. Various features of each pattern may analyzed and extracted,such as a pattern location within a die, a shape, a size, a density, aneighborhood layout, a pattern type, etc.

Further, the plurality of design patterns may be classified intodifferent categories based on the extracted features. As illustrated inFIGS. 5A-5C, a subset of patterns with a similar or same shape may begrouped together via pattern grouping. For example, a first subset ofpatterns, Group 1, may include patterns with a same or similar shape topattern 501 a. For example, a second subset of patterns, Group 2, mayinclude patterns with a same or similar shape to pattern 501 b. Forexample, a third subset of patterns, Group 3, may include patterns witha same or similar shape to pattern 501 c. Each pattern group may beassociated with corresponding metadata, which may include information ofa pattern location within a die, pattern type, shape, size and otherextracted features.

The automated SEM sampling system may comprise a smart inspectionsampling planner (e.g., an inspection sampling planner component). Thesmart inspection sampling planner may identify a plurality of traininglocations on a sample of the product to use in relation to training themachine learning model based on the analyzing results of the analyzer.The GDS database of the product may have information regarding alocation associated with each pattern group. Thus, the design patternsrendered from the GDS may contain location information. Therefore, byanalyzing and recognizing pattern groups from the GDS, locations ofcorresponding pattern groups on a wafer of the product may bedetermined.

FIG. 5D is a diagram 500 d illustrating a plurality of traininglocations 506 t on a wafer 503 (e.g. 403, described in connection withFIG. 4 ). For each pattern group, there are many potential locations 506to acquire training images, as illustrated in FIG. 5D. The automated SEMsampling system may be further configured to determine one or morespecific training locations 506 t for obtaining training images. Forexample, the SEM sampling system may identify the one or more traininglocations based on one or more of a location within a die, an inspectionarea, a field of view (FOV), or other imaging parameters such as localalignment points (LAPs) and auto-focus points on the covered area in thewafer, from the analyzing results of the analyzer. For example, for eachpattern group, the SEM sampling system may determine the one or moretraining locations based on, at least in part, a location within a die.A planner may automatically generate die sampling across the wafer.

A scanning path may be analyzed and created based on the overall scanareas for all the pattern groups. The scanning path may be determined bya parameter such as location, FOV, etc., or some combination of these.Furthermore, the scanning path along with other parameters, such as FOV,shape, type, etc., may be used according to a recipe for an electronbeam tool. The electron beam tool may be configured to follow the recipeto automatically scan and capture training images for the machinelearning module. For example, LAPs and auto-focus points may bedetermined based on the factors such as a number of field of views(FOVs) and a distance between each FOV, etc.

FIG. 6A is a block diagram 600 a illustrating a flow diagram of a systemfor automatically obtaining training images 610 to train a machinelearning model 615 that improves image quality, according to someaspects of the present disclosure. FIG. 6B is a block diagram 600 billustrating details of an automatic SEM sampling system, according tosome aspects of the present disclosure. Referring to FIG. 6A and FIG.6B, the method implemented by the system may be performed by anautomated SEM sampling system 602 (e.g., a processor 604, the computersystem 309) communicating with an EBI tool 612 (e.g., the EBI system300). For example, the automated SEM sampling system may be a controllerof the EBI tool, and the controller may have circuitry to cause the EBItool to perform the method. For example, the method may includeperforming pattern analysis and classification as by a GDS analyzer 603(e.g., a GDS analyzer component 603 of the processor). The patternanalysis and classification may be performed based on various features,for example, line pattern, logic pattern, 1D/2D pattern, dense/isolatedpattern, etc. The method may further comprise grouping same or similarpatterns together via pattern grouping.

The method may comprise, by a sampling planner 605 (e.g., a samplingplanner component 605 of the processor), determining scan areas, whichare training locations, based on the analyzing results of the step ofanalyzing. The analyzing results of the step of analyzing may includepattern locations with a die, inspection areas, FOV sizes, and LAPpoints and auto-focus points and other imaging parameters based on thecovered area in the wafer.

The method may comprise, by a user interface 608, enabling scanning eachof the training locations a plurality of times to create a plurality oftraining images 610 for each training location, and obtaining theplurality of training images 610 from the EBI tool 612. For example,some of the plurality of images may be low quality images, e.g.,generated using a low number of scans. For example, some of theplurality of images may have enhanced image quality, e.g., generatedusing an increased number of scans. The enhanced image quality may referto a higher resolution, a higher contrast, a higher sensitivity, ahigher brightness, or a lower noise level, etc. For example, some of thetraining images 610 may reflect an improvement in image quality thatresults from the additional number of scans of a training location. Insome aspects, a plurality of low quality image and high quality imagepairs may be obtained, via the user interface 608, and be used astraining images 610 to train the machine learning model 615. Forexample, a low quality SEM imaging mode may be based on default settingor user-input throughput requirements. For example, a high quality SEMimage mode may be based on default setting or user-input qualityrequirements. In some aspects, a user may also have the option ofdirectly inputting high quality reference image 611. For example, thehigh quality reference images 611 may be stored in a storage medium 606.In such cases, acquisition of high quality images may be skipped.

The method may further comprise using the machine learning model 615(e.g., a machine learning model component of the processor) to modify animage to approximate a result obtained with an increased number ofscans. Various machine learning methods can be employed in the machinelearning model 615 to learn the enhancement framework from the trainingimage pairs 610. The machine learning model 615 may be parametric. Datamay be collected for the machine learning model 615.

A quality enhancement module 617 (e.g., a quality enhancement module 617of the processor) may be learned at the end of the step of using themachine learning model for each type of pattern-of-interest. The qualityenhancement module 617 can be used directly for inspection or metrologypurpose in a high throughput mode. After being trained based on imagessampled from the automatic sampling system 602, the quality enhancementmodule 617 may be used for image enhancement without training data.Therefore, the quality enhancement module 617 may be anon-parameterized, which does not involve the use of an excessive numberof parameter settings that may result in too much overhead. Accordingly,the quality enhancement module 617 is advantageous to generate highquality images with high throughput, thereby increasing manufacturingefficiency and reducing manufacturing cost.

FIG. 7 is a flowchart 700 illustrating an example of a method ofautomatically obtaining training images to train a machine learningmodel that improves image quality, according to some aspects of thepresent disclosure. The method may be performed by an automated SEMsampling system (e.g., 102, 202, 602) communicating with an EBI tool(e.g., 212, 612). For example, the automated SEM sampling system may bea controller of the EBI tool, and the controller may have circuitry tocause the EBI tool to perform the method.

As shown in FIG. 7 , at step 702, the method may comprise analyzing aplurality of patterns of data relating to a layout of a product toidentify a plurality of training locations on a sample of the product touse in relation to training the machine learning model. For example, thedata may be in a database. For example, the database may be any one of agraphic database system (GDS), an Open Artwork System InterchangeStandard, or a Caltech Intermediate Form, among others. For example, theGDS may include both GDS and GDSII.

For example, the step of analyzing the plurality of patterns of datarelating to layout of the product may further comprise classifying theplurality of patterns into a plurality of subsets of patterns. Forexample, the step of analyzing a plurality of patterns of data relatingto layout of a product further may comprise extracting a feature fromthe plurality of patterns. For example, the classifying the plurality ofpatterns into a plurality of subsets of patterns may be based on theextracted feature. For example, each subset of the plurality of subsetsof patterns may be associated with information relating to a location, atype, a shape, a size, a density or a neighborhood layout. For example,identifying the plurality of training locations may be based on a fieldof view, a local alignment point, or an auto-focus point. For example,identifying the plurality of training locations may comprise identifyingone or more training locations for each subset of patterns.

At step 704, the method may comprise obtaining a first image having afirst quality for each of the plurality of training locations. Forexample, the step of obtaining a first image having a first quality foreach of the plurality of training locations comprises obtaining morethan one first image having a first quality for each of the plurality oftraining locations.

For example, the method may further comprise determining a firstscanning path including a first scan for obtaining the first image. Forexample, the first scanning path may be based on an overall scan areafor the plurality of training locations. For example, the first scanningpath may be determined by some of the parameters such as location, FOV,etc. Furthermore, the first scanning path along with other parameters,such as FOV, shape, type, etc., may provide a first recipe to anelectron beam tool. The electron beam tool may be configured to followthe first recipe to automatically scan and capture images for themachine learning module.

At step 706, the method may comprise obtaining a second image having asecond quality for each of the plurality of training locations. Forexample, the second quality may be higher than the first quality. Forexample, the step of obtaining a second image having a second qualityfor each of the plurality of training locations comprises obtaining morethan one second image having a second quality for each of the pluralityof training locations.

For example, the method may comprise determining a second scanning pathincluding a second scan for obtaining the second image. For example, thesecond scanning path based on an overall scan area for the plurality oftraining locations. For example, the second scanning path may bedetermined by some of the parameters such as location, FOV, etc.Furthermore, the second scanning path along with other parameters, suchas FOV, shape, type, etc., may provide a second recipe to the electronbeam tool. The electron beam tool may be configured to follow the secondrecipe to automatically scan and capture images for the machine learningmodule. For example, the first scan may include a first number of scansand the second scan may include a second number of scans, where thesecond number of scans may be larger than the first number of scans.

For example, the second image may be obtained as a reference image by anoptional user input.

At step 708, the method may comprise using the first image and thesecond image to train the machine learning model.

At step 710, the method may comprise using the machine learning model tomodify an image to approximate a result obtained with an increasednumber of scans.

For example, the method may further comprise using the machine learningmodel to modify a first image of a location to obtain a second image ofthe location, where the second image has an enhanced quality than thefirst image. In this way, the method is advantageous to obtain highquality images with high throughput, thereby increasing manufacturingefficiency and reduce manufacturing cost. Further, the method are fullyautomatic. Thus, the method may prevent human error and inconsistencyfrom different operators. therefore, the method is further advantageousto increase inspection accuracy.

Now referring back to FIG. 2 , the computer system 202 may be acontroller of inspection system 212 (e.g., e-beam inspection system) andthe controller may include circuitry for: analyzing a plurality ofpatterns of data relating to a layout of a product to identify aplurality of training locations on a sample of the product to use inrelation to training the machine learning model; obtaining a first imagehaving a first quality for each of the plurality of training locations;obtaining a second image having a second quality for each of theplurality of training locations, the second quality higher than thefirst quality; and using the first image and the second image to trainthe machine learning model.

Further referring to FIG. 2 , the storage medium 206 may be anon-transitory computer readable medium storing a set of instructionsthat is executable by a controller of a device to cause the device toperform a method comprising: analyzing a plurality of patterns of datarelating to a layout of a product to identify a plurality of traininglocations on a sample of the product to use in relation to training themachine learning model; obtaining a first image having a first qualityfor each of the plurality of training locations; obtaining a secondimage having a second quality for each of the plurality of traininglocations, the second quality higher than the first quality; and usingthe first image and the second image to train the machine learningmodel.

The embodiments may further be described using the following clauses:

1. A method of automatically obtaining training images for use intraining a machine learning model, the method comprising:

analyzing a plurality of patterns of data relating to layout of aproduct to identify a plurality of training locations to use in relationto training the machine learning model;

obtaining a first image having a first quality for each of the pluralityof training locations;

obtaining a second image having a second quality for each of theplurality of training locations, the second quality being higher thanthe first quality; and

using the first image and the second image to train the machine learningmodel.

2. The method of clause 1 wherein the data is in a database.

3. The method of clause 2 wherein the database is any one of a graphicdatabase system (GDS), an Open Artwork System Interchange Standard, or aCaltech Intermediate Form.

4. The method of clause 3 wherein the GDS includes GDS formatted data orGDSII formatted data.

5. The method of clause 1 wherein the step of obtaining a first imagehaving a first quality for each of the plurality of training locationscomprises obtaining more than one first image having a first quality foreach of the plurality of training locations.6. The method of clause 1 wherein the step of obtaining a second imagehaving a second quality for each of the plurality of training locationscomprises obtaining more than one second image having a second qualityfor each of the plurality of training locations.7. The method of clause 1, wherein the step of analyzing the pluralityof patterns of data relating to layout of the product further comprisesclassifying the plurality of patterns into a plurality of subsets ofpatterns.8. The method of any one of clauses 1 to 7, wherein the step ofanalyzing a plurality of patterns of data relating to layout of aproduct further comprises extracting a feature from the plurality ofpatterns.9. The method of clause 8, wherein the extracted feature includes ashape, a size, a density, or a neighborhood layout.10. The method of clause 7 wherein the classifying the plurality ofpatterns into a plurality of subsets of patterns is based on theextracted feature.11. The method of clause 7 wherein each subset of the plurality ofsubsets of patterns is associated with information relating to alocation, a type, a shape, a size, a density or a neighborhood layout.12. The method of any one of clauses 1 to 11, wherein identifying theplurality of training locations is based on a field of view, a localalignment point, or an auto-focus point.13. The method of any one of clauses 1 to 12, wherein the method furthercomprises determining

a first scanning path including a first scan for obtaining the firstimage, the first scanning path based on an overall scan area for theplurality of training locations.

14. The method of clause 13, wherein the method further comprises

determining a second scanning path including a second scan for obtainingthe second image, the second scanning path based on an overall scan areafor the plurality of training locations.

15. The method of clause 14, wherein the first scan includes a firstnumber of scans, wherein the second scan includes a second number ofscans, and wherein the second number of scans is larger than the firstnumber of scans.

16. The method of any one of clauses 1 to 13, wherein the second imageis obtained as a reference image by an optional user input.

17. The method of any one of clauses 1 to 16, further comprising usingthe machine learning model to modify a first image of a location toobtain a second image of the location, wherein the second image has anenhanced quality than the first image.

18. The method of any one of clauses 1 to 17, wherein identifying theplurality of training locations comprises identifying one or moretraining locations for each subset of patterns.

19. The method of any one of clauses 1 to 18, wherein the qualityincludes a resolution, a contrast, a brightness, or a noise level.

20. The method of any one of clauses 1 to 19, further comprising

using the machine learning model to modify an image to approximate aresult obtained with an increased number of scans.

21. An apparatus for automatically obtaining training images to train aML model that improves image quality, the apparatus comprising:

a memory; and

at least one processor coupled to the memory and configured to:

-   -   analyze a plurality of patterns of data relating to a layout of        a product to identify a plurality of training locations to use        in relation to training the machine learning model;    -   obtain a first image having a first quality for each of the        plurality of training locations;    -   obtain a second image having a second quality for each of the        plurality of training locations, the second quality being higher        than the first quality; and    -   use the first image and the second image to train the machine        learning model.        22. The apparatus of clause 21 wherein the data is in a        database.        23. The apparatus of clause 22, wherein the database is any one        of a graphic database system (GDS), an Open Artwork System        Interchange Standard, or a Caltech Intermediate Form.        24. The apparatus of clause 23, where the GDS includes GDS        formatted data or GDSII formatted data.        25. The apparatus of clause 21 wherein the at least one        processor is further configured to obtain more than one first        image having a first quality for each of the plurality of        training locations.        26. The apparatus of clause 21, wherein the at least one        processor is further configured to obtain more than one second        image having a second quality for each of the plurality of        training locations.        27. The apparatus of clause 21, wherein the at least one        processor is further configured to classify the plurality of        patterns into a plurality of subsets of patterns.        28. The apparatus of clause 21, wherein the at least one        processor is further configured to extract a feature from the        plurality of patterns.        29. The apparatus of clause 28, wherein the extracted feature        includes a shape, a size, a density, or a neighborhood layout.        30. The apparatus of clause 27, wherein the at least one        processor is further configured to classify the plurality of        patterns into a plurality of subsets of patterns based on the        extracted feature.        31. The apparatus of clause 27, wherein each subset of the        plurality of subsets of patterns is associated with information        relating to a location, a type, a shape, a size, a density or a        neighborhood layout.        32. The apparatus of any one of clauses 21 to 31, wherein the at        least one processor is further configured to identify the        plurality of training locations based on a field of view, a        local alignment point, or an auto-focus point.        33. The apparatus of any one of clauses 21 to 32, wherein the at        least one processor is further configured to

determine a first scanning path including a first scan for obtaining thefirst image, the first scanning path based on an overall scan area forthe plurality of training locations.

34. The apparatus of clause 33, wherein the at least one processor isfurther configured to

determine a second scanning path including a second scan for obtainingthe second image, the second scanning path based on an overall scan areafor the plurality of training locations.

35. The apparatus of clause 34, wherein the first scan includes a firstnumber of scans, wherein the second scan includes a second number ofscans, and wherein the second number of scans is larger than the firstnumber of scans.

36. The apparatus of any one of clauses 21 to 33, wherein the secondimage is received as a reference image by an optional user input.

37. The apparatus of any one of clauses 21 to 36, wherein the at leastone processor is further configured to

use the machine learning model to modify a first image of a location toobtain a second image of the location, wherein the second image has anenhanced quality than the first image.

38. The apparatus of any one of clauses 21 to 37, wherein the at leastone processor is further configured to

identify one or more training locations for each subset of patterns.

39. The apparatus of any one of clauses 21 to 38, wherein the qualityincludes a resolution, a contrast, a brightness, or a noise level.

40. The apparatus of any one of clauses 21 to 39, wherein the at leastone processor is further configured to

use the machine learning model to modify an image to approximate aresult obtained with an increased number of scans.

41. A non-transitory computer readable medium storing a set ofinstructions that is executable by a controller of a device to cause thedevice to perform a method comprising:

analyzing a plurality of patterns of data relating to a layout of aproduct to identify a plurality of training locations to use in relationto training the machine learning model;

obtaining a first image having a first quality for each of the pluralityof training locations;

obtaining a second image having a second quality for each of theplurality of training locations, the second quality being higher thanthe first quality; and

using the first image and the second image to train the machine learningmodel.

42. The non-transitory computer readable medium of clause 41 wherein thedata is in a database.

43. The non-transitory computer readable medium of clause 42 wherein thedatabase is any one of a graphic database system (GDS), an Open ArtworkSystem Interchange Standard, a Caltech Intermediate Form, or ElectronicDesign Interchange Format.

44. The non-transitory computer readable medium of clause 43 where theGDS includes at least one of GDS or GDSII.

45. The non-transitory computer readable medium of clause 41 wherein thestep of obtaining a first image having a first quality for each of theplurality of training locations further comprises obtaining more thanone first image having a first quality for each of the plurality oftraining locations.46. The non-transitory computer readable medium of clause 41, whereinthe step of obtaining a second image having a second quality for each ofthe plurality of training locations comprises obtaining more than onesecond image having a second quality for each of the plurality oftraining locations.47. The non-transitory computer readable medium of clause 41, whereinthe step of analyzing the plurality of patterns of data relating tolayout of the product further comprises classifying the plurality ofpatterns into a plurality of subsets of patterns.48. The non-transitory computer readable medium of any one of clauses 41to 47, wherein the step of analyzing a plurality of patterns of datarelating to layout of a product further comprises extracting a featurefrom the plurality of patterns.49. The non-transitory computer readable medium of clause 48, whereinthe extracted feature includes a shape, a size, a density, or aneighborhood layout.50. The non-transitory computer readable medium of clause 47, whereinthe classifying the plurality of patterns into a plurality of subsets ofpatterns is based on the extracted feature.51. The non-transitory computer readable medium of clause 47, whereineach subset of the plurality of subsets of patterns is associated withinformation relating to a location, a type, a shape, a size, a densityor a neighborhood layout.52. The non-transitory computer readable medium of any one of clauses 41to 51, wherein identifying the plurality of training locations is basedon a field of view, a local alignment point, or an auto-focus point.53. The non-transitory computer readable medium of any one of clauses 41to 52, wherein the method further comprises

determining a first scanning path including a first scan for obtainingthe first image, the first scanning path based on an overall scan areafor the plurality of training locations.

54. The non-transitory computer readable medium of clause 53, whereinthe method further comprises

determining a second scanning path including a second scan for obtainingthe second image, the second scanning path based on an overall scan areafor the plurality of training locations.

55. The non-transitory computer readable medium of clause 54, whereinthe first scan includes a first number of scans, wherein the second scanincludes a second number of scans, and wherein the second number ofscans is larger than the first number of scans.56. The non-transitory computer readable medium of any one of clauses 41to 53, wherein the second image is obtained as a reference image by anoptional user input.57. The non-transitory computer readable medium of any one of clauses 41to 56, wherein the method further comprises using the machine learningmodel to modify a first image of a location to obtain a second image ofthe location, wherein the second image has an enhanced quality than thefirst image.58. The non-transitory computer readable medium of any one of clauses 41to 57, wherein identifying the plurality of training locations comprisesidentifying one or more training locations for each subset of patterns.59. The non-transitory computer readable medium of any one of clauses 41to 58, wherein the quality includes a resolution, a contrast, abrightness, or a noise level.60. The non-transitory computer readable medium of any one of clauses 41to 59, wherein the method further comprises

using the machine learning model to modify an image to approximate aresult obtained with an increased number of scans.

61. An electron beam inspection apparatus, comprising:

a controller having circuitry to cause the electron beam inspectionapparatus to perform:

-   -   analyzing a plurality of patterns of data relating to a layout        of a product to identify a plurality of training locations to        use in relation to training the machine learning model;    -   obtaining a first image having a first quality for each of the        plurality of training locations;    -   obtaining a second image having a second quality for each of the        plurality of training locations, the second quality being higher        than the first quality; and    -   using the first image and the second image to train the machine        learning model.        62. The electron beam inspection apparatus of clause 61 wherein        the data is in a database.        63. The electron beam inspection apparatus of clause 62 wherein        the database is any one of a graphic database system (GDS), an        Open Artwork System Interchange Standard, or a Caltech        Intermediate Form.        64. The electron beam inspection apparatus of clause 63 where        the GDS includes at least one of GDS or GDSII.        65. The electron beam inspection apparatus of clause 61 wherein        the step of obtaining a first image having a first quality for        each of the plurality of training locations comprises obtaining        more than one first image having a first quality for each of the        plurality of training locations.        66. The electron beam inspection apparatus of clause 61 wherein        the step of obtaining a second image having a second quality for        each of the plurality of training locations comprises obtaining        more than one second image having a second quality for each of        the plurality of training locations.        67. The electron beam inspection apparatus of clause 61, wherein        the step of analyzing the plurality of patterns of data relating        to layout of the product further comprises classifying the        plurality of patterns into a plurality of subsets of patterns.        68. The electron beam inspection apparatus of any one of clauses        61 to 67, wherein the step of analyzing a plurality of patterns        of data relating to layout of a product further comprises        extracting a feature from the plurality of patterns.        69. The electron beam inspection apparatus of clause 68, wherein        the extracted feature includes a shape, a size, a density, or a        neighborhood layout.        70. The electron beam inspection apparatus of clause 67, wherein        the classifying the plurality of patterns into a plurality of        subsets of patterns is based on the extracted feature.        71. The electron beam inspection apparatus of clause 67, wherein        each subset of the plurality of subsets of patterns is        associated with information relating to a location, a type, a        shape, a size, a density or a neighborhood layout.        72. The electron beam inspection apparatus of any one of clauses        61 to 71, wherein identifying the plurality of training        locations is based on a field of view, a local alignment point,        or an auto-focus point.        73. The electron beam inspection apparatus of any one of clauses        61 to 72, wherein the controller having circuitry to cause the        electron beam inspection apparatus to further perform:

determining a first scanning path including a first scan for obtainingthe first image, the first scanning path based on an overall scan areafor the plurality of training locations.

74. The electron beam inspection apparatus of clause 73, wherein thecontroller having circuitry to cause the electron beam inspectionapparatus to further perform:

determining a second scanning path including a second scan for obtainingthe second image, the second scanning path based on an overall scan areafor the plurality of training locations.

75. The electron beam inspection apparatus of clause 74, wherein thefirst scan includes a first number of scans, wherein the second scanincludes a second number of scans, and wherein the second number ofscans is larger than the first number of scans.

76. The electron beam inspection apparatus of any one of clauses 61 to73, wherein the second image is obtained as a reference image by anoptional user input.

77. The electron beam inspection apparatus of any one of clauses 61 to76, wherein the controller having circuitry to cause the electron beaminspection apparatus to further perform:

using the machine learning model to modify a first image of a locationto obtain a second image of the location, wherein the second image hasan enhanced quality than the first image.

78. The electron beam inspection apparatus of any one of clauses 61 to77, wherein identifying the plurality of training locations comprisesidentifying one or more training locations for each subset of patterns.

79. The electron beam inspection apparatus of any one of clauses 61 to78, wherein the quality includes a resolution, a contrast, a brightness,or a noise level.

80. The electron beam inspection apparatus of any one of clauses 61 to79, wherein the controller having circuitry to cause the electron beaminspection apparatus to further perform:

using the machine learning model to modify an image to approximate aresult obtained with an increased number of scans.

81. The method of clause 1, wherein the patterns of data are scanningelectron microscope (SEM) images of the product.

82. The method of clause 1, wherein the second image is obtained basedon a plurality of low quality images.

83. The method of clause 1, wherein the second image is obtained byaveraging the plurality of low quality images.

84. The method of clause 1, wherein the second image is obtained bycombining the plurality of low quality images.

85. The method of clause 1, wherein the first image and the second imageare images obtained by one or more scanning electron microscopes, andwherein the second image is a higher quality image than the first image.

86. The method of clause 85, wherein the second image has a higherresolution, higher contrast, higher brightness, or reduced noise levelas compared to the first image.

87. The apparatus of clause 21, wherein the patterns of data arescanning electron microscope (SEM) images of the product.

88. The apparatus of clause 21, wherein the second image is obtainedbased on a plurality of low quality images.

89. The apparatus of clause 21, wherein the second image is obtained byaveraging the plurality of low quality images.

90. The apparatus of clause 21, wherein the second image is obtained bycombining the plurality of low quality images.

91. The non-transitory computer readable medium of clause 41, whereinthe patterns of data are scanning electron microscope (SEM) images ofthe product.

92. The non-transitory computer readable medium of clause 41, whereinthe second image is obtained based on a plurality of low quality images.

93. The non-transitory computer readable medium of clause 41, whereinthe second image is obtained by averaging the plurality of low qualityimages.

94. The non-transitory computer readable medium of clause 41, whereinthe second image is obtained by combining the plurality of low qualityimages.

95. The electron beam inspection apparatus of clause 61, wherein thepatterns of data are scanning electron microscope (SEM) images of theproduct.

96. The electron beam inspection apparatus of clause 61, wherein thesecond image is obtained based on a plurality of low quality images.

97. The electron beam inspection apparatus of clause 61, wherein thesecond image is obtained by averaging the plurality of low qualityimages.

98. The electron beam inspection apparatus of clause 61, wherein thesecond image is obtained by combining the plurality of low qualityimages.

Example aspects or embodiments are described above with reference toflowchart illustrations or block diagrams of methods, apparatus(systems) and computer program products. It will be understood that eachblock of the flowchart illustrations or block diagrams, and combinationsof blocks in the flowchart illustrations or block diagrams, can beimplemented by computer program product or instructions on a computerprogram product. These computer program instructions may be provided toa processor of a computer, or other programmable data processingapparatus to produce a machine, such that the instructions, whichexecute via the processor of the computer or other programmable dataprocessing apparatus, create means for implementing the functions/actsspecified in the flowchart or block diagram block or blocks.

These computer program instructions may also be stored in a computerreadable medium that can direct a hardware processor core of a computer,other programmable data processing apparatus, or other devices tofunction in a particular manner, such that the instructions stored inthe computer readable medium form an article of manufacture includinginstructions which implement the function/act specified in the flowchartor block diagram block or blocks.

The computer program instructions may also be loaded onto a computer,other programmable data processing apparatus, or other devices to causea series of operational steps to be performed on the computer, otherprogrammable apparatus or other devices to produce a computerimplemented process such that the instructions which execute on thecomputer or other programmable apparatus provide processes forimplementing the functions/acts specified in the flowchart or blockdiagram block or blocks.

Any combination of one or more computer readable medium(s) may beutilized. The computer readable medium may be a non-transitory computerreadable storage medium. A computer readable storage medium may be, forexample, but is not limited to, an electronic, magnetic, optical,electromagnetic, infrared, or semiconductor system, apparatus, ordevice, or any suitable combination of the foregoing. More specificexamples (a non-exhaustive list) of the computer readable storage mediumwould include the following: an electrical connection having one or morewires, a portable computer diskette, a hard disk, a random access memory(RAM), a read-only memory (ROM), an erasable programmable read-onlymemory (EPROM, EEPROM or Flash memory), an optical fiber, a cloudstorage, a portable compact disc read-only memory (CD-ROM), an opticalstorage device, a magnetic storage device, or any suitable combinationof the foregoing. In the context of this document, a computer readablestorage medium may be any tangible medium that can contain or store aprogram for use by or in connection with an instruction executionsystem, apparatus, or device.

Program code embodied on a computer readable medium may be transmittedusing any appropriate medium, including but not limited to wireless,wireline, optical fiber cable, RF, IR, etc., or any suitable combinationof the foregoing.

Computer program code for carrying out operations for exampleembodiments may be written in any combination of one or more programminglanguages, including an object-oriented programming language such asJava, Smalltalk, C++ or the like and conventional procedural programminglanguages, such as the “C” programming language or similar programminglanguages. The program code may execute entirely on the user's computer,partly on the user's computer, as a stand-alone software package, partlyon the user's computer and partly on a remote computer or entirely onthe remote computer or server. In the latter scenario, the remotecomputer may be connected to the user's computer through any type ofnetwork, including a local area network (LAN) or a wide area network(WAN), or the connection may be made to an external computer (forexample, through the Internet using an Internet Service Provider).

The flowchart and block diagrams in the Figures illustrate examples ofthe architecture, functionality, and operation of possibleimplementations of systems, methods and computer program productsaccording to various embodiments. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof code, which comprises one or more executable instructions forimplementing the specified logical function(s). It should also be notedthat, in some alternative implementations, the functions noted in theblock may occur out of the order noted in the figures. For example, twoblocks shown in succession may, in fact, be executed substantiallyconcurrently, or the blocks may sometimes be executed in the reverseorder, depending upon the functionality involved. It will also be notedthat each block of the block diagrams or flowchart illustration, andcombinations of blocks in the block diagrams or flowchart illustration,can be implemented by special purpose hardware-based systems thatperform the specified functions or acts, or combinations of specialpurpose hardware and computer instructions.

It is understood that the described embodiments are not mutuallyexclusive, and elements, components, materials, or steps described inconnection with one example embodiment may be combined with, oreliminated from, other embodiments in suitable ways to accomplishdesired design objectives.

Reference herein to “some aspects”, “some embodiments” or “someexemplary embodiments” mean that a particular feature, structure, orcharacteristic described in connection with the embodiment can beincluded in at least one aspect, or one embodiment. The appearance ofthe phrases “one aspect”, “some aspects”, “one embodiment”, “someembodiments” or “some exemplary embodiments” in various places in thespecification do not all necessarily refer to the same embodiment, norare separate or alternative embodiments necessarily mutually exclusiveof other embodiments.

It should be understood that the steps of the example methods set forthherein are not necessarily required to be performed in the orderdescribed, and the order of the steps of such methods should beunderstood to be merely example. Likewise, additional steps may beincluded in such methods, and certain steps may be omitted or combined,in methods consistent with various embodiments.

As used herein, unless specifically stated otherwise, the term “or”encompasses all possible combinations, except where infeasible. Forexample, if it is stated that a component may include A or B, then,unless specifically stated otherwise or infeasible, the component mayinclude A, or B, or A and B. As a second example, if it is stated that acomponent may include A, B, or C, then, unless specifically statedotherwise or infeasible, the component may include A, or B, or C, or Aand B, or A and C, or B and C, or A and B and C.

As used in this application, the word “exemplary” is used herein to meanserving as an example, instance, or illustration. Any aspect or designdescribed herein as “exemplary” is not necessarily to be construed aspreferred or advantageous over other aspects or designs. Rather, use ofthe word is intended to present concepts in a concrete fashion.

Additionally, the articles “a” and “an” as used in this application andthe appended claims should generally be construed to mean “one or more”unless specified otherwise or clear from context to be directed to asingular form.

Unless explicitly stated otherwise, each numerical value and rangeshould be interpreted as being approximate as if the word “about” or“approximately” preceded the value of the value or range.

The use of figure numbers or figure reference labels in the claims isintended to identify one or more possible embodiments of the claimedsubject matter to facilitate the interpretation of the claims. Such useis not to be construed as necessarily limiting the scope of those claimsto the embodiments shown in the corresponding figures.

Although the elements in the following method claims, if any, arerecited in a particular sequence with corresponding labeling, unless theclaim recitations otherwise imply a particular sequence for implementingsome or all of those elements, those elements are not necessarilyintended to be limited to being implemented in that particular sequence.

It will be further understood that various changes in the details,materials, and arrangements of the parts which have been described andillustrated in order to explain the nature of described aspects orembodiments may be made by those skilled in the art without departingfrom the scope as expressed in the following claims.

The invention claimed is:
 1. An apparatus for automatically obtainingtraining images to train a machine learning model that improves imagequality, the apparatus comprising: a memory; and at least one processorcoupled to the memory and configured to: analyze a plurality of patternsof data relating to a layout of a product to identify a plurality oftraining locations to use in relation to training the machine learningmodel; obtain a first scanning image having a first quality for each ofthe plurality of training locations; obtain a second scanning imagehaving a second quality for each of the plurality of training locations,the second quality being higher than the first quality; and use thefirst scanning image and the second scanning image to train the machinelearning model.
 2. The apparatus of claim 1, wherein the data is in adatabase.
 3. The apparatus of claim 2, wherein the database is any oneof a graphic database system (GDS), an Open Artwork System InterchangeStandard, or a Caltech Intermediate Form.
 4. The apparatus of claim 3,where the GDS includes GDS formatted data or GDSII formatted data. 5.The apparatus of claim 1 wherein the at least one processor is furtherconfigured to obtain more than one first scanning image having a firstquality for each of the plurality of training locations.
 6. Theapparatus of claim 1, wherein the at least one processor is furtherconfigured to obtain more than one second scanning image having a secondquality for each of the plurality of training locations.
 7. Theapparatus of claim 1, wherein the at least one processor is furtherconfigured to classify the plurality of patterns into a plurality ofsubsets of patterns.
 8. The apparatus of claim 1, wherein the at leastone processor is further configured to extract a feature from theplurality of patterns.
 9. The apparatus of claim 8, wherein theextracted feature includes a shape, a size, a density, or a neighborhoodlayout.
 10. The apparatus of claim 7, wherein the at least one processoris further configured to classify the plurality of patterns into aplurality of subsets of patterns based on the extracted feature.
 11. Theapparatus of claim 7, wherein each subset of the plurality of subsets ofpatterns is associated with information relating to a location, a type,a shape, a size, a density or a neighborhood layout.
 12. The apparatusof claim 1, wherein the at least one processor is further configured toidentify the plurality of training locations based on a field of view, alocal alignment point, or an auto-focus point.
 13. The apparatus ofclaim 1, wherein the at least one processor is further configured todetermine a first scanning path including a first scan for obtaining thefirst scanning image, the first scanning path based on an overall scanarea for the plurality of training locations.
 14. The apparatus of claim13, wherein the at least one processor is further configured todetermine a second scanning path including a second scan for obtainingthe second scanning image, the second scanning path based on an overallscan area for the plurality of training locations.
 15. A non-transitorycomputer readable medium storing a set of instructions that isexecutable by a controller of a device to cause the device to perform amethod comprising: analyzing a plurality of patterns of data relating toa layout of a product to identify a plurality of training locations touse in relation to training the machine learning model; obtain a firstscanning image having a first quality for each of the plurality oftraining locations; obtain a second scanning image having a secondquality for each of the plurality of training locations, the secondquality being higher than the first quality; and use the first scanningimage and the second scanning image to train the machine learning model.16. A method of automatically obtaining training images for use intraining a machine learning model, the method comprising: analyzing aplurality of patterns of data relating to a layout of a product toidentify a plurality of training locations to use in relation totraining the machine learning model; obtaining a first scanning imagehaving a first quality for each of the plurality of training locations;obtaining a second scanning image having a second quality for each ofthe plurality of training locations, the second quality being higherthan the first quality; and using the first scanning image and thesecond scanning image to train the machine learning model.